from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.014	0.066	0.0	0.018	1	5	0.744	2.406	0.034	7.489	0.110	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.014	0.066	0.0	0.018	1	5	0.744	0.025	0.001	720.556	19.410	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.344	0.017	0.0	0.344	1	5	1.000	2.406	0.034	0.143	0.007	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.344	0.017	0.0	0.344	1	5	1.000	0.025	0.001	13.748	0.778	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.151	0.115	0.0	0.018	-1	100	0.836	3.010	0.070	6.031	0.146	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.151	0.115	0.0	0.018	-1	100	0.836	0.028	0.003	647.964	68.322	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.336	0.014	0.0	0.336	-1	100	0.000	3.010	0.070	0.112	0.005	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.336	0.014	0.0	0.336	-1	100	0.000	0.028	0.003	11.994	1.358	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.995	0.150	0.0	0.018	-1	1	0.657	2.204	0.084	8.165	0.319	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.995	0.150	0.0	0.018	-1	1	0.657	0.026	0.003	693.156	67.640	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.331	0.013	0.0	0.331	-1	1	1.000	2.204	0.084	0.150	0.008	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.331	0.013	0.0	0.331	-1	1	1.000	0.026	0.003	12.739	1.339	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.096	0.005	0.0	0.018	1	100	0.836	2.408	0.035	7.516	0.109	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.096	0.005	0.0	0.018	1	100	0.836	0.024	0.002	739.304	48.035	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.340	0.009	0.0	0.340	1	100	0.000	2.408	0.035	0.141	0.004	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.340	0.009	0.0	0.340	1	100	0.000	0.024	0.002	13.905	0.972	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.242	0.221	0.0	0.018	1	1	0.657	1.591	0.027	11.467	0.239	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.242	0.221	0.0	0.018	1	1	0.657	0.026	0.002	704.936	53.969	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.315	0.011	0.0	0.315	1	1	1.000	1.591	0.027	0.198	0.008	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.315	0.011	0.0	0.315	1	1	1.000	0.026	0.002	12.191	1.010	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.097	0.021	0.0	0.018	-1	5	0.744	3.026	0.065	5.981	0.129	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.097	0.021	0.0	0.018	-1	5	0.744	0.029	0.003	627.899	57.309	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.328	0.011	0.0	0.328	-1	5	1.000	3.026	0.065	0.108	0.004	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.328	0.011	0.0	0.328	-1	5	1.000	0.029	0.003	11.374	1.108	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.198	0.040	0.0	0.004	1	5	0.883	2.142	0.041	1.960	0.041	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.198	0.040	0.0	0.004	1	5	0.883	0.003	0.000	1440.725	104.945	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.277	0.009	0.0	0.277	1	5	1.000	2.142	0.041	0.129	0.005	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.277	0.009	0.0	0.277	1	5	1.000	0.003	0.000	94.939	7.545	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.215	0.075	0.0	0.004	-1	100	0.906	2.715	0.064	1.552	0.046	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.215	0.075	0.0	0.004	-1	100	0.906	0.011	0.003	398.894	101.355	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.275	0.010	0.0	0.275	-1	100	1.000	2.715	0.064	0.101	0.004	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.275	0.010	0.0	0.275	-1	100	1.000	0.011	0.003	26.038	6.672	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.193	0.093	0.0	0.004	-1	1	0.874	1.816	0.028	2.309	0.062	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.193	0.093	0.0	0.004	-1	1	0.874	0.009	0.002	464.992	104.468	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.012	0.0	0.261	-1	1	0.000	1.816	0.028	0.144	0.007	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.012	0.0	0.261	-1	1	0.000	0.009	0.002	28.948	6.611	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.107	0.052	0.0	0.004	1	100	0.906	2.144	0.042	1.915	0.045	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.107	0.052	0.0	0.004	1	100	0.906	0.003	0.000	1443.035	158.546	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.020	0.0	0.276	1	100	1.000	2.144	0.042	0.129	0.010	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.020	0.0	0.276	1	100	1.000	0.003	0.000	96.872	12.648	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.129	0.051	0.0	0.004	1	1	0.874	1.276	0.022	3.236	0.070	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.129	0.051	0.0	0.004	1	1	0.874	0.002	0.000	1803.555	189.793	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.262	0.010	0.0	0.262	1	1	0.000	1.276	0.022	0.205	0.009	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.262	0.010	0.0	0.262	1	1	0.000	0.002	0.000	114.355	12.777	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.088	0.075	0.0	0.004	-1	5	0.883	2.715	0.067	1.505	0.046	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.088	0.075	0.0	0.004	-1	5	0.883	0.010	0.003	395.917	112.979	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.256	0.016	0.0	0.256	-1	5	1.000	2.715	0.067	0.094	0.006	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.256	0.016	0.0	0.256	-1	5	1.000	0.010	0.003	24.814	7.239	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	205.159	0.000	0.0	0.205	-1	1	0.955	0.531	0.045	386.651	33.031	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	205.159	0.000	0.0	0.205	-1	1	0.955	0.004	0.001	58504.793	12398.151	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.973	0.322	0.0	2.973	-1	1	1.000	0.531	0.045	5.602	0.773	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.973	0.322	0.0	2.973	-1	1	1.000	0.004	0.001	847.679	201.717	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	222.566	0.000	0.0	0.223	-1	5	0.973	1.009	0.048	220.680	10.458	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	222.566	0.000	0.0	0.223	-1	5	0.973	0.005	0.002	47485.065	21592.569	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.926	0.202	0.0	2.926	-1	5	1.000	1.009	0.048	2.901	0.243	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.926	0.202	0.0	2.926	-1	5	1.000	0.005	0.002	624.274	287.112	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	225.490	0.000	0.0	0.225	1	5	0.973	1.641	0.029	137.448	2.450	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	225.490	0.000	0.0	0.225	1	5	0.973	0.002	0.001	118868.819	33655.051	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.903	0.110	0.0	2.903	1	5	1.000	1.641	0.029	1.770	0.074	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.903	0.110	0.0	2.903	1	5	1.000	0.002	0.001	1530.460	437.155	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	248.023	0.000	0.0	0.248	-1	100	0.969	3.362	0.064	73.776	1.403	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	248.023	0.000	0.0	0.248	-1	100	0.969	0.008	0.001	31165.963	5126.639	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.942	0.182	0.0	2.942	-1	100	1.000	3.362	0.064	0.875	0.057	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.942	0.182	0.0	2.942	-1	100	1.000	0.008	0.001	369.674	64.947	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	245.788	0.000	0.0	0.246	1	100	0.969	5.642	0.080	43.561	0.617	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	245.788	0.000	0.0	0.246	1	100	0.969	0.006	0.002	44589.360	12417.081	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.944	0.101	0.0	2.944	1	100	1.000	5.642	0.080	0.522	0.019	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.944	0.101	0.0	2.944	1	100	1.000	0.006	0.002	534.009	149.839	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	252.221	0.000	0.0	0.252	1	1	0.955	0.868	0.023	290.411	7.758	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	252.221	0.000	0.0	0.252	1	1	0.955	0.002	0.000	159000.362	38075.287	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.958	0.202	0.0	2.958	1	1	1.000	0.868	0.023	3.406	0.250	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.958	0.202	0.0	2.958	1	1	1.000	0.002	0.000	1864.943	464.364	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.003	0.0	0.000	-1	1	0.888	0.028	0.002	1.478	0.135	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.003	0.0	0.000	-1	1	0.888	0.003	0.001	14.131	3.817	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	1	1.000	0.028	0.002	0.213	0.030	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	1	1.000	0.003	0.001	2.034	0.590	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.044	0.002	0.0	0.000	-1	5	0.912	0.032	0.003	1.381	0.150	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.044	0.002	0.0	0.000	-1	5	0.912	0.003	0.000	14.795	2.472	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	5	1.000	0.032	0.003	0.181	0.030	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	5	1.000	0.003	0.000	1.945	0.407	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.049	0.004	0.0	0.000	1	5	0.912	0.028	0.002	1.788	0.218	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.049	0.004	0.0	0.000	1	5	0.912	0.001	0.000	66.070	22.363	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	5	1.000	0.028	0.002	0.214	0.030	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	5	1.000	0.001	0.000	7.904	2.729	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.068	0.005	0.0	0.000	-1	100	0.898	0.047	0.007	1.443	0.240	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.068	0.005	0.0	0.000	-1	100	0.898	0.003	0.001	20.294	6.575	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	100	1.000	0.047	0.007	0.126	0.026	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	100	1.000	0.003	0.001	1.772	0.611	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.064	0.004	0.0	0.000	1	100	0.898	0.042	0.007	1.513	0.268	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.064	0.004	0.0	0.000	1	100	0.898	0.001	0.000	72.562	16.466	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	100	1.000	0.042	0.007	0.139	0.034	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	1	100	1.000	0.001	0.000	6.679	1.876	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.002	0.0	0.000	1	1	0.888	0.023	0.001	1.918	0.138	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.002	0.0	0.000	1	1	0.888	0.001	0.000	58.496	16.784	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.002	0.0	0.006	1	1	1.000	0.023	0.001	0.244	0.066	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.002	0.0	0.006	1	1	1.000	0.001	0.000	7.433	2.893	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.537	0.027	0.001	0.001	0.832	0.195	0.007	2.754	0.168	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.537	0.027	0.001	0.001	0.832	0.024	0.003	22.085	2.595	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.424	0.017	0.000	0.424	1.000	0.195	0.007	2.171	0.112	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.424	0.017	0.000	0.424	1.000	0.024	0.003	17.406	1.965	See	See